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Concept maps are visual representations of knowledge 
structure and thought. Not enough attention has been paid to the measurement 
and influence of preservice teachers ' knowledge structure and its development 
over time. This study implemented instructional scaffolding interventions in 
order to determine the effect on preservice teachers' knowledge structure 
(i.e., concept maps) and short essay responses. The study was also designed 
to develop a reliable method of measuring knowledge structure and to describe 
the characteristics of knowledge structure over time and between groups. 
Twenty preservice teachers enrolled at a state-supported university in the 
Midwest participated in the study. Participants were enrolled in one of two 
"Introduction to Educational Psychology" courses during the spring of 1998. A 
two-group, multi -variate repeated measures design was employed. Concept maps 
and short essay responses were compared over time. The findings suggest that 
instructional scaffolding interventions have, over time, a positive effect on 
students' ability to: create broad categories with subsuming structures; 
apply key terminology; construe interpretable patterns; decipher the demands 
of the task; reference the content discussed; and ultimately, produce 
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Measurement and Influence of Preservice Teachers’ Knowledge Structure in an 

Educational Psychology Course 

ABSTRACT 

Concept maps are visual representations of knowledge structure and thought. While it is 
sensible to assume that structured schemata are related to skillful essay responses, not 
enough attention has been paid to the measurement and influence of preservice 
teachers’ knowledge structure and its development over time. Instructional scaffolding 
interventions (ISI) were implemented in order to determine its effect on preservice 
teachers’ knowledge structure (i.e., concept maps) and essay responses. Twenty 
preservice teachers enrolled at a state-supported university in the mid-west participated 
in the study. Participants were enrolled in one of two Introduction to Educational 
Psychology courses that were taught by the investigator during the spring term of 1998. 
A two-group, multivariate repeated measures design was employed. A univariate 
analysis revealed a significant different between groups in phase three F (1, 18) = 6.57, 
N = 20; power = .679, Sig. = .020. The findings suggest that instructional scaffolding 
interventions have, over time, a positive effect on students' ability to: create broad 
categories with subsuming structures, apply key terminology, construe interpretable 
patterns, decipher the demands of the task, and ultimately, produce superior essays. 
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Measurement and Influence of Preservice Teachers’ Knowledge Structure in an 

Educational Psychology Course. 



OBJECTIVES 

The objectives of this study were: 1) to determine the effect of instructional 
scaffolding interventions (ISI) on students’ knowledge structures and short essay 
responses; 2) to develop a reliable method to measure knowledge structure; and 3) to 
describe the characteristics of knowledge structure over time and between groups. 

PERSPECTIVES and THEORETICAL FRAMEWORK 

Teacher educators have realized that consideration of instructional methods 
should take into account the way in which knowledge is initially organized and 
represented in students and the ways in which their knowledge representations are 
modified by new information. If organization is the issue, then we need a way of 
measuring organization - and we need to influence it. 

Measurement of Knowledge Structure 

Several researchers have developed measures to infer students’ knowledge 
structures and their development in college courses (e.g., Naveh-Benjamin, McKeachie, 
et. al. , 1986; Winitzky, 1992; Winitzky, Kauchak & Kelly, 1994). Numerous studies to 
measure knowledge structure involve the use of Reitman and Rueter’s (1980) ordered 
tree technique which involves transforming the data into distance matrices from a 
standard cognitive structure. For example, Strahan (1989) used the ordered tree 
technique to examine experienced and novice middle-school teachers’ views of 
instruction. It was found that although the experienced and novice teachers selected 
many of the same terms, the experienced teachers: 1 ) used more terms in constructing 
their ordered trees; 2) organized them into more chunks; and 3) created more linkages 
between chunks (Strahan, 1989). Naveh-Benjamin, McKeachie, et. al., (1986) modified 
the ordered tree technique to generate four measures of cognitive structure - grouping, 
hierarchical structure, directionality of the structure, and similarity to the structure. 

A problem with the research so far is that most of the studies use general 
distance from a standard structure (an ordered tree, or hierarchy) as the measure of 
structure development. The assumption that single concepts or sets of concepts are 
mentally organized into an ordered tree (i.e., hierarchy) impedes other patterns of 
knowledge structure. It is held that there is a need to develop a measurement of 
knowledge structure that accommodates other structural possibilities (e.g., sequence, 
cause-effect, cycle, descriptive, compare/contrast). 
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Scaffolding as Part of Classroom Instruction 

Given that the studies previously mentioned add to our knowledge that structure 
changes as a result of learning, we also need to determine what instructional strategies 
can help to influence knowledge structure. Scaffolding is an instructional technique that 
provides assistance and allows students to complete tasks they are not able to 
complete independently (Wood, Bruner, & Ross, 1976). In Becoming a Scaffolder of 
Students' Learning, Hogan and Pressley (1997) suggest a number of demonstrative 
actions to support scaffolding efforts with students. Without in-depth description, they 
are as follows: 

• Use a Socratic style of interaction 

• Provide feedback, but avoid directly evaluating students’ thinking 

• Model thinking processes 

• Provide explanation as necessary 

• Provide tailored assistance 

• • Encourage and capitalize on students’ comments and questions 

• Respond flexibly to students’ errors 

• Maintain an atmosphere that supports intellectual risk 

METHODOLOGY 

Participants 

Twenty preservice teachers enrolled at a public, state-supported university in the 
mid-west participated in the study. Participants attended one of two Introduction to 
Educational Psychology courses that were taught by the investigator during the spring 
term of 1 998. 

Design 

A two-group, multivariate repeated measures design was employed. Concept 
maps and short essay responses were compared over time (i.e. , three phases of the 
study) in intervention and non-intervention classroom settings. The independent 
variables were: phases of study (X1a-X3a); and groups (X1b-X2b). The dependent 
variables were quality of concept maps (Y1, Y3, Y5) and written expressions of 
conceptual understanding (Y2, Y4, Y6). The analytic paradigm is illustrated in Figure 1. 

Independent Variables 

a) Groups . Two groups were established: experimental and control. 

b) Phases of Study . There were three phases of the investigation. Each phase 
consisted of 3 to 4 class sessions. 
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Dependent Variables 

The following dependent measures were used: 

a) A concept map representation sheet was used to assess the quality of 
students’ knowledge structure of motivation. Concept map work sheets were 
administered to all participants in each phase of the study. Seven criteria were used to 
infer the characteristics of the knowledge structure: 1 ) inclusiveness of content - the 
degree to which motivational theories were included in the concept map; 2) explanatory 
focus - the degree to which the concept map explained student motivation; 3) fluency - 
the degree to which key terms were used in the concept map; 4) breadth - the degree 
to which broad categories were used in the concept map; 5) depth of categorization - 
the degree to which subsuming terms were used to describe each category; 6) 
interpretability - the degree to which the structure of the concept map is understood or 
brings about meaning; and 7) originality - the degree to which the structure or design 
was used by fewer than 20% of the participants. The investigator assigned a score on a 
five-point scale (1 - 5) to each of the criteria listed above. A score of five points on any 
criteria indicated excellence; three points - satisfactory work; one point - needs 
improvement. (Four points and two points were assigned for work that appeared to fall 
in between the extreme categories and the mid-point). A perfect map received thirty-five 
points. 

b) A short essay question sheet was designed to assess the participants’ written 
expressions of conceptual understanding of motivation theory. Only three criteria were 
used to judge the quality of essay responses in each phase of the study: 1) 
inclusiveness of content; 2) explanatory focus; and 3) fluency. A score on a five-point 
scale (1-5) was assigned to each of the criteria described above. A perfect essay 
received a fifteen point total score. 

Most of the .intercorrelations between the criteria to assess concept maps and 
essays were significant at the 0.01 level (2-tailed) (see Tables 1 and 2). In addition, a 
Pearson Product correlation coefficient revealed a positive relationship between the 
concept map and essay scores (r = .599, £> < .01, N = 20, 2-tailed) (see Table 3 and 
Figure 2). 

Procedures 

Experimental Group . ISI followed individual mapping and writing activities (see 
Appendix A for examples of concept maps and short essay responses). ISI consisted 
as a series of guided information feedback sessions to support the learning of struggling 
students. Students volunteered to give knowledge to help other students develop an 
appropriate schematic representation of motivation theory. A six-page packet containing 
a prior selection of students’ concept maps of motivation theory were handed-out to 
each member of the class. The packet contained a representative collection of low- 
quality to high-quality conceptualizations of human motivation. In small groups, students 
were instructed to provide written feedback regarding the quality of the map structures 
contained in the packet (see Appendix B for examples of students’ concept maps with 
other students’ feedback comments). Each group was then asked to create a 
“collaborative concept map” to form a new concept map (see Appendix C for examples 
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of collaborative concept maps). Each group presented their newly configured and 
collaboratively determined concept map to the whole class using an overhead projector. 
Ideally, through this collaborative activity, the “social” or “community” support in the 
class would lead to better patterns of text organization, and short essay responses. The 
investigator and voluntary class members took the role of scaffolds in providing 
guidance to other class members. The goal was to communicate to others the 
information that was relevant and/or irrelevant to the task. Concept map and short 
essay responses were collected within a 4-week period. 

RESULTS AND CONCLUSIONS 

As expected, group 1 mean score (map and essay combined) was greater than 
group 2, in each phase of the study (see Table 4 and Figure 3). A MANOVA on the 
dependent variables did not reveal an overall significant difference between groups 
(Wilks’ Lambda value = .700, F = 2.29, N = 20; df = 16, g < .05). However, a univariate 
analysis revealed a significant different between groups in phase three F (1, 18) = 6.57, 
N = 20; power = .679, Sig. = .020 (see Table 5). 

Concept Map Variable . Mean total concept map score during phase three was 
higher for group 1 = 27.30, than group 2 = 22.10. For group 1, the originality index 
decreased more, indicating that students’ knowledge structures became more similar to 
that of the other students as a result of social learning. 

For both groups, mean score for inclusiveness of the content and fluency was 
higher than explanatory focus, breadth and depth of categorization criteria (see figures 
4 and 5). Furthermore, breadth and depth of categorization criteria showed the greatest 
improvement throughout the unit. 

Essay Variable . Mean total essay score in phase three was higher for group 1 = 
10.90, than group 2 = 4.90. For both groups, mean score for inclusiveness of the 
content was higher than explanatory focus and fluency criteria (see figures 6 and 7). 

Two Extreme Cases 

Two participants, one with a highly structured concept map and one with a low- 
structured concept map were selected for closer scrutiny to illustrate qualitatively the 
nature of the quantitative findings by comparing extreme cases. 

The concept map in Figure 8 shows how participant #1, conceived of the 
motivation theory, as presented from the text and classes. This participant earned a 
total map score of 11 (inclusiveness to the content, 0; depth of the categories, 0; 
explanatory focus, 0; fluency, 0; breadth, 4; depthness, 0; interpretability, 2; originality, 
5. In this sequential pattern, events are arranged in a chronological order that has a 
specific beginning and end. Clearly this pattern has “big ideas” and does not focus 
attention on key elements of motivation theory. 

On the other hand, participant #2’s concept map (refer to Figure 9) earned a 
total map score of 29 (inclusiveness to the content, 5; explanatory focus, 5; fluency, 5; 
breadth, 5; depthness, 4; interpretability, 4; and originality, 1. This hierarchical pattern 
includes a main concept and the subconcepts under it. That is, the general idea of 
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“explaining motivation” and the four theories, or subcategories of it. Moreover, key terms 
are added under each theory to explain motivation. 

EDUCATIONAL IMPORTANCE OF THE WORK 

The present findings suggest that instructional scaffolding interventions increase 
students’ ability to formulate broad categorizations, subsuming structures, key 
terminology, interpretable patterns, to reference the content discussed, and to remain 
on task (i.e., to answer the question). At the same time, scaffolding interventions tend to 
decrease original pattern structure as the struggling learner moves through the zone of 
proximal development to build better knowledge structures independently. 

Given that knowledge structure and short essay responses were moderately 
correlated, these findings direct our attention to a major problem in writing activities. 
Namely, inappropriate development of overall relations and a general framework for the 
theoretical matter learned, which further prevents assimilation of new information (i.e., 
accretion), restructuring, and tuning. The use of ISI as an independent variable provides 
conclusive findings that the positive relationship between knowledge structure and 
written expressions of conceptual understanding is causal. 

The criteria developed to measure knowledge structure and essay responses 
indicate satisfaction. However, further research is necessary to shed light on construct 
related evidence of validity, and issues of reliability. The initial reason for developing this 
measure was the feeling that conventional achievement tests measured bits of 
knowledge but were inadequate measures of the students’ organization of knowledge. 
The addition of this method to the array of other methods for measuring structure and 
with other methods to analyze specific knowledge (e.g., essay) should provide us with a 
more meaningful assessment and representation of knowledge structure. This is a 
major aid toward an understanding of knowledge structure and development and a 
framework for designing strategies for teaching. 

Finally, a critical gap in our understanding of the link between preservice 
teacher’s knowledge structure and their actual behavior. If we can help teachers better 
organize what they know, will that enable them to better utilize that knowledge at the 
appropriate time and place? 
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Figure 2. Correlation Between Total Map 
and Essay Scores For All Phases (N = 20) 
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Figure 1 . Analytic Paradigm. 
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Figure 3. Mean total score for experimental 
and comparison group during each phase. 
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Figure 4. Group 1 Mean Criteria Scores (Map) 
in Phase 3 (N=10). 
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Figure 5. Group 1 Mean Criteria Scores (Essay) 

in Phase 3 (N=10). 
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Figure 6. Group 2 Mean Criteria Scores (Map) 
in Phase 3 (N=10). 
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Figure 7. Group 2 Mean Criteria Scores (Essay) 

in Phase 3 (N=10). 
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Table 1 



Intercorrelations Between Criteria Used to Evaluate Concept Maps (N = 20) 



Correlations 





m. breadth 


m.dept 

hn 


m.expl 

an 


m.flue 

n 


m.incl 

us 


m.int 

erpr 


m. origin 


Pearson m. breadth 


1.000 


.079 


.482* 


.508* 


.482* 


.463* 


.014 


Correlation m.depthn 


.079 


1.000 


.723** 


.757** 


.605** 


.535* 


-.539* 


m. explan 


.482* 


.723** 


1.000 


.885** 


.806** 


.552* 


-.286 


m.fiuen 


.508* 


.757** 


.885** 


1.000 


.888** 


.572** 


-.371 


m.inclus 


.482* 


.605** 


.806** 


.888** 


1.000 


.545* 


-.112 


m.interpr 


.463* 


.535* 


.552* 


.572** 


.545* 


1.000 


.067 


m. origin 


.014 


-.539* 


-.286 


-.371 


-.112 


.067 


1.000 


Sig. m. breadth 




.739 


.032 


.022 


.031 


.040 


.955 


(2-tailed) m.depthn 


.739 




.000 


.000 


.005 


.015 


.014 


m. explan 


.032 


.000 




.000 


.000 


.012 


.222 


m.fiuen 


.022 


.000 


.000 




.000'- 


.008 


.108 


m.inclus 


.031 


.005 


.000 


.000 




.013 


.639 


m.interpr 


.040 


.015 


.012 


.008 


.013 




.780 


m. origin 


.955 


.014 


.222 


.108 


.639 


.780 




N m. breadth 


20 


20 


20 


20 


20 


20 


20 


m.depthn 


20 


20 


20 


20 


20 


20 


20 


m. explan 


20 


20 


20 


20 


20 


20 


20 


m.fiuen 


20 


20 


20 


20 


20 


20 


20 


m.inclus 


20 


20 


20 


20 


20 


20 


20 


m.interpr 


20 


20 


20 


20 


20 


20 


20 


m. origin 


20 


20 


20 


20 


20 


20 


20 



*. Correlation is significant at the 0.05 level (2-tailed). 
**■ Correlation is significant at the 0.01 level (2-taiied). 
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Table 2 



Intercorrelations Between Criteria to Evaluate Written Essays (N = 20) 



Correlations 





w.explan 


w.fluen 


W.inclus 


Pearson 


w. explan 


1.000 


.553* 


.481* 


Correlation 


w.fluen 


.553* 


1.000 


.918" 




w.inclus 


.481* 


.918** 


1.000 


Sig. 


w.explan 




.011 


.032 


(2-tailed) 


w.fluen 


.011 




.000 




w.inclus 


.032 


.000 




N 


w.explan 


20 


20 


20 




w.fluen 


20 


20 


20 




w.inclus 


20 


20 


20 



* Correlation is significant at the 0.05 level (2-tailed). 
** Correlation is significant at the 0.01 level (2-tailed). 




BEST COPY AVAILABLE 



18 



Table 3 



Correlation Between Total Concept Map and Total Written Essay Scores. 



Correlations 





TotalMap 


totalW 


Pearson 

Correlation 


TotalMap 
total W 


1.000 

.599** 


.599~ 

1.000 


Sig. 

(2-tailed) 


TotalMap 

totalW 


’ .005 


.005 


N 


TotalMap 


20 


20 




totalW 


20 


20 



**. Correlation is significant at the 0.01 level (2-tailed). 



Figure 2. Correlation Between Total Map 
and Essay Scores For All Phases (N = 20). 
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Table 4 



Mean Total Scores For Experimental and Comparison 
Groups for All Phases 



Report 



Group 


YITot 

al 

Points 


Y2 

Total 

Poin 

ts 


TOT 

Y1Y2 


Y3 

Total 


Y4 

Total 


TOT 

Y3Y 

4 


Y5T 

OTAL 


Y6T 

otal 


TotY5 

Y6 


Group 1 


Mean 


17.30 


6.00 


23.30 


24.00 


6.70 


30.6 


27.30 


11 


38.20 




N 


10 


10 


10 


10 


10 


10 


10 


10 


10 




Std. 

Deviation 


6.36 


2.94 


8.18 


8.21 


2.67 


10.1 


5.85 


4.46 


9.03 


Group 2 


Mean 


15.30 


5.50 


20.80 


20.00 


7.70 


27.7 


22.10 


4.90 


27.00 




N 


10 


10 


10 


10 


10 


10 


10 


10 


10 




Std. 

Deviation 


5.03 


2.01 


5.07 


10.89 


3.13' 


11.8 


5.67 


5.69 


10.47 


Total 


Mean 


16.30 


5.75 


22.05 


22.00 


7.20 


29.2 


24.70 


7.90 


32.60 




N 


20 


20 


20 


20 


20 


20 


20 


20 


20 




Std. 

Deviation 


5.68 


2.47 


6.75 


9.61 


2.88 


10.8 


6.21 


5.85 


11.11 
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Table 5 



MANOVA Between Groups for Each Phase 



Between-Subjects Factors 





Value 






Label 


N 


' Group 1 


Group 1 


10 


2 


Group 2 


10 



Multivariate Tests c 

















Noncen 












Hypot 






t. 


Observ 










hesis 






Param 


ed 


Effect 




Value 


F 


df 


Error df 


Sig. 


eter 


Power 3 


Intercept 


Pillars 

Trace 


. 940 


83.903 b 


3.000 


16.000 


.000 


251.709 


1.000 




Wilks' 

Lambda 


.060 


83.903 b 


3.000 


16.000 


.000 


251.709 


1.000 




Hotelling's 

Trace 


15.732 


83.903 b 


3.000 


16.000 


.000 


251.709 


1.000 




Roy's 




K 














Largest 

Root 


15.732 


83.903 


3.000 


16.000 


.000 


251.709 


1.000 


GROUP 


Pillars 

Trace 


.300 


2.290 b 


3.000 


16.000 


.117 


6.871 


.472 




Wilks' 

Lambda 


.700 


Z290 b 


3.000 


16.000 


.117 


6.871 


.472 




Hotelling's 
T race 


.429 


2.290 b 


3.000 


16.000 


.117 


6.871 


.472 




Roy's 




u 














Largest 

Root 


.429 


2.290 


3.000 


16.000 


.117 


6.871 


.472 



a. Computed using alpha = .05 



b. Exact statistic 

c. Design: Intercept+GROUP 
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Tests of Between-Subjects Effects 



Source 


Depende 
nt Variable 


Type III 
Sum of 
Squares 


df 


Mean 

Square 


F 


Sig. 


Noncent. 

Parameter 


Observ 

ed 

Power* 


Corrected 


TOTY1Y2 


31.250 b 


1 


31.250 


.675 


.422 


.675 


.122 


Model . 


TOTY3Y4 


42.050° 


1 


42.050 


.347 


.563 


.347 


.086 




TotY5Y6 


627.200 d 


1 


627.200 


6.565 


.020 


6.565 


.679 


Intercept 


TOTY1Y2 


9724.050 


1 


9724.050 


209.95 


.000 


209.947 


1.000 




TOTY3Y4 


16994.45 


1 


16994.45 


140.29 


.000 


140.289 


1.000 




TotY5Y6 


21255.20 


1 


21255.20 


‘222.49 


.000 


222.490 


1.000 


GROUP 


TOTY1Y2 


31.250 


1 


31.250 


.675 


.422 


.675 


.122 




TOTY3Y4 


42.050 


1 


42.050 


.347 


.563 


.347 


.086 




TotY5Y6 


627.200 


1 


627.200 


6.565 


.020 


6.565 


.679 


Error 


TOTY1Y2 


833.700 


18 


46.317 












TOTY3Y4 


2180.500 


18 


121.139 












TotY5Y6 


1719.600 


18 


95.533 










Total 


TOTY1Y2 


10589.00 


20 














TOTY3Y4 


19217.00 


20 














TotY5Y6 


23602.00 


20 












Corrected 


TOTY1Y2 


864.950 


19 












Total 


TOTY3Y4 


2222,550 


19 














TotY5Y6 


2346.800 


19 













a. Computed using alpha = .05 



b. R Squared = .036 (Adjusted R Squared = -.017) 

c. R Squared = .019 (Adjusted R Squared = -.036) 

d. R Squared = .267 (Adjusted R Squared = .227) 
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APPENDIX A 



Scenario : Other teachers marvel at how adept you are in motivating students to 
learn in your 11 th grade English class. As such, you have been asked by your 
Principal to present an all-day workshop on Student Motivation for parents, 
teachers and administrators within your school district. As part of the workshop, 
you would like to present an advance organizer to introduce different theoretical 
explanations of motivation. 

Based on Motivation theories discussed in class, 
create a concept map to explain human motivation according to different 

theoretical points of view. 
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Although Bill and Elliot are fairly similar in ability, they are as different as night and 
day in their approaches to school. Elliot seems to care only about how he looks to 
others. For the school science fair, for example, he selected an extremely easy 
project on ants that looked fancy and impressive but, actually, have very little 
substance. He continually makes excuses before taking tests, such as “I’m taking this 
without any studying.” Or he might say, “I’ll be happy with a C." After the test, 
however, he is quick to promote his own good performances when they occur. Bill is 
quite the opposite. He loves challenges and becomes totally immersed in books and 
projects. He says that when he gets interested in something, he wants to learn all 
that he can. He usually earns good grades in his courses, but he seems basically 
unconcerned about his tests. 

Based on Motivation theories discussed in class, explain t he behaviors of the two 
boys according to different theoretical points of view . 
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